PatchUp: A Feature-Space Block-Level Regularization Technique for Convolutional Neural Networks

Large capacity deep learning models are often prone to a high generalization gap when trained with limited amount of labeled training data. A recent class methods address this problem uses various ways construct new sample by mixing pair (or more) samples. We propose PatchUp, hidden state block-level regularization technique for Convolutional Neural Networks (CNNs), that is applied on selected contiguous blocks feature maps from random Our approach improves the robustness CNN against manifold intrusion may occur in other state-of-the-art approaches. Moreover, since we block features space, which has more dimensions than input obtain diverse samples towards different dimensions. experiments CIFAR10/100, SVHN, Tiny-ImageNet, and ImageNet using ResNet architectures including PreActResnet18/34, WRN-28-10, ResNet101/152 show PatchUp upon, or equals, performance current regularizers CNNs. also can provide better deformed robust adversarial attacks.